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ABSTRACT 

This  report  reviews  the  progress  made  during  FY87  on  the  Defense 
Nuclear  Agency (DNA)  sponsored  Remote  Security  Station  (RSS). 

During  this  time  frame,  Sandia  National  Laboratories  was  tasked 
to  develop  a  Phase  1  proof-of-principle  system  consisting  of 
readily  available  hardware.  The  primary  emphasis  was  placed  upon 
development  of  software  and  sensor  fusion  techniques,  and  as  a 
result  the  Phase  1  hardware  is  not  suitable  for  field  deployment. 

The  RSS  consists  of  a  portable  (non-mobile)  sensor  platform  and 
an  intelligent  controller.  The  paper  includes  descriptions  of 
the  sensor  platform  and  control  console  hardware,  as  well  as  the 
software  that  runs  the  intelligent  controller.  The  sensor  fusion 
techniques  originally  considered  are  discussed,  and  the  methods 
used  in  the  current  system  are  explained  in  detail.  Phase  2 
development  of  the  RSS  is  scheduled  for  FY88U  and  a  discussion  of  i 
the  planned  improvements  is  included  •  i,  ye  V.-  "o rs, 
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1 .  INTRODUCTION 

The  purpose  of  the  RSS  project,  which  began  in  mid-FY87,  is 
to  develop  a  robotic  sensor  platform  to  enhance  a  fixed-site 
security  control  center  by  providing  localized  detection  and 
assessment.  The  intent  is  (1)  to  improve  detection,  primarily 
by  reducing  the  false  alarm  rate  (FAR)  through  sensor  fusion  and 
alarm  analysis  techniques,  and  (2)  to  improve  assessment  by 
cueing  the  operator  with  the  location  of  the  threat.  Phase  1 
concentrated  on  developing  sensor  fusion  techniques  rather  than 
hardware.  The  goal  was  to  demonstrate  in  October  1987  a  "Proof 
of  Principle"  (POP)  Remote  Security  Station  using  laboratory 
hardware.  A  field  prototype  RSS  will  be  built  in  Phase  2,  and  it 
is  anticipated  that  the  Phase  2  sensor  suite  will  be  incorporated 
on  a  mobile  platform  in  Phase  3. 

Copies  of  the  viewgraphs  used  for  the  project  review  are  included 
in  Appendix  A. 

Applications  of  this  type  of  robotic  sensor  platform  for  fixed- 
site  security  include  use  at  both  temporary  and  permanent 
locations,  and  use  as  a  mobile  sensor  platform.  A  portable  plat¬ 
form  could  be  used  for  temporary  enhancement  or  replacement  of 
existing  Intrusion  Detection  Systems  (IDSs) .  For  instance,  if  an 
intrusion  is  likely  at  a  particular  point,  or  if  some  part  of  the 
IDS  is  isolated  and  vulnerable,  a  portable  platform  could  augment 
an  existing  system  or  be  positioned  to  replace  faulty  components 
of  an  existing  IDS  until  they  are  repaired.  Another  situation 
where  a  portable  RSS  would  be  beneficial  is  to  protect  high-value 
assets  that  are  present  only  for  short  periods  of  time — for 
instance,  if  an  aircraft  or  special  transporter  is  parked  where 
installation  of  permanent  IDS  sensors  would  be  impractical. 

Examples  of  permanent  locations  of  a  robotic  sensor  platform 
include  usinq  it  to  cover  specific  high-priority  locations  in 
a  hardened  fixed  configuration,  on  a  tower,  or  in  an  unobtrusive 
pop-up  shelter. 

Mobile  applications  would  offer  even  more  benefit  from  a  robotic 
sensor  platform.  A  teleoperated  or  autonomous  mobile  sensor 
platform  could  provide  greater  detection  and  assessment  capabil¬ 
ities  than  a  stationary  platform  because  of  its  ability  to  cover 
larger  areas.  A  vehicle-mounted  RSS  that  could  autonomously 
patrol  certain  areas  of  a  fixed  site  would  free  operators  at  the 
security  control  center  for  other  tasks  until  an  alarm  required 
their  direct  attention.  If  the  platform  included  delay  or 
deterrent  devices,  these  devices  could  be  safely  activated  from 
the  security  control  center  without  placing  personnel  in 
potential  danger. 
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The  major  components  in  this  POP  Remote  Security  Station  include 
the  following: 

o  Sensors  for  detection,  assessment,  and  environmental 
monitoring. 

o  A  portable  (nonmobile)  pan/tilt  sensor  platform,  or  pod, 
which  can  be  teleoperated  manually  or  computer  controlled, 
o  A  nonportable  intelligent  controller,  located  in  a  security 
control  center  that  includes  processors  and  the  operator's 
interface. 

o  A  communications  link  consisting  of  individual  cables  for 
transmitting  power  and  data  between  the  sensor  platform  and 
the  intelligent  controller. 

o  Sensor  fusion  methods  that  include  sensor  combination,  alarm 
prioritizing,  assessment  cuing,  and  image  processing,  with 
emphasis  on  alarm  prioritizing  and  image  processing. 

2.  DESCRIPTION  OF  THE  HARDWARE 

To  meet  a  tight  schedule,  the  Phase  1  system  was  assembled  with 
common,  off-the-shelf  hardware  that  had  short  delivery  times,  was 
available  on  loan,  or  was  left  over  from  inactive  programs.  The 
POP  system  is  not  suitable  for  field  use  since  it  is  ineffi¬ 
ciently  packaged  and  not  weatherproof.  The  sensors  in  this 
laboratory  prototype  system  are  intended  to  be  representative  of 
those  that  might  be  required  in  a  fixed-site  application,  and  are 
used  here  to  demonstrate  sensor  fusion  techniques.  Figure  1  is  a 
block  diagram  of  the  Phase  1  RSS  system. 

Sensors 

Both  security  and  environmental  sensors  were  included  in  the 
Phase  1  RSS. 

The  security  sensors  include: 

o  Cohu  B/W  CCD  TV  camera  used  for  both  detection  (with  the 
Video  Motion  Detector  (VMD) ,  described  below)  and 
assessment . 

o  Eltec  Passive  Infrared  Motion  Sensor  (PIMS) ,  also  called 
an  infrared  (IR)  telescope  because  of  its  narrow  field-of- 
view  (2  degrees) ,  used  for  detection. 

o  Sandia-developed  acoustic  array  that  identifies  the  bear¬ 
ing  angle  of  acoustic  targets  (see  Appendix  B  for 
details) . 


o  Bionic  Ear  directional  microphone  used  for  assessment. 


DNA  SYSTEM  BLOCK  DIAGRAM 
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o  Infrared  (IR)  spotlight  that  provides  a  night-vision 

capability  by  taking  advantage  of  the  CCD  camera's  ability 
to  operate  in  the  near-IR  region.  This  device  provides 
illumination  for  the  ccd  camera  while  remaining  invisible 
to  the  naked  eye.  (This  is  the  only  active  device  on  the 
RSS . ) 

Other  security  sensors  considered  were  a  forward-looking  infrared 
imager  (FLIR) ,  seismic  disturbance  detector,  monostatic  microwave 
motion  sensor,  laser  range  finder,  acoustic  helicopter  detector 
(SHAD) ,  and  a  Ground  Surveillance  Radar  (GSR) .  Arrangements  were 
made  through  DNA  to  borrow  a  GSR  for  evaluation. 

Sandia  National  Laboratories  (SNL) ,  Exploratory  Systems  Develop¬ 
ment  Division  9133,  has  developed  image-processing  algorithms 
that  alert  to  meaningful  motion  occurring  in  a  video  image.  The 
image  can  be  generated  by  several  types  of  imaging  devices  such 
as  a  TV  camera,  thermal  or  laser  imagers.  The  RSS  uses  a  black- 
and-white  CCD  camera  as  input  to  this  VMD  software.  However,  it 
only  functions  when  the  sensor  platform  is  stationary,  and  for 
motion  in  the  sensitized  portions  of  the  camera's  f ield-of-view 
as  set  up  by  the  operator. 

The  environmental  sensors  incorporated  in  the  laboratory  hardware 
RSS  measure  light  level,  temperature,  humidity,  and  wind.  They 
were  part  of  the  Sensor  Environment  Monitor  (SEM) ,  which  was 
developed  to  monitor  the  environment  of  interior  intrusion  detec¬ 
tion  sensors.  Therefore,  these  sensors  were  never  intended  to 
witness  the  extremes  of  exterior  environments  and  are  not 
weatherproof. 

Remote  Sensor  Platform 

The  remote  sensor  platform,  shown  in  Figure  2,  consists  of  intru¬ 
sion  sensors  mounted  on  a  pan/tilt  platform  supported  by  a 
tripod,  three  acoustic  array  microphones  located  120  degrees 
apart  on  the  ground  about  1  meter  from  the  center  of  the  tripod, 
the  environmental  sensors,  and  the  pod  controller  placed  nearby. 
Cables  from  the  controller  supply  the  necessary  110  VAC  power  to 
the  pod. 

Controller 

The  intelligent  controller  will  be  located  at  a  facility  security 
control  center  where  normal  110  VAC  power  is  available.  The  POP 
system  is  housed  in  two  slope-front  racks  with  a  desk  top  panel. 
The  operator's  control  console  (Figure  3)  contains  a  video 
monitor,  digitizing  tablet  and  graphics  display,  host  computer, 
keyboard,  and  computer  display.  All  instructions  and  controls 
are  input  through  the  keyboard  except  for  audio  speaker  volume, 

IR  spotlight  controls,  and  TV  camera  lens  zoom  and  focus.  Two 
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speakers  provide  audio  from  the  directional  microphone  and  one 
of  the  acoustic  array  microphones.  An  IBM/XT  is  the  heart  of 
the  intelligent  controller.  VME  chassis  used  for  the  acoustic 
targeting  and  the  Video  Motion  Detection  (VMD)  image-processing 
systems  are  also  housed  in  the  racks. 

3.  SENSOR  FUSION 

Introduction 

Sensor  fusion  applied  to  intrusion  sensors  should  reduce  false 
alarm  rates  (FAR)  and  improve  the  probability  of  detection  (PD) . 
It  can  be  approached  in  many  different  ways.  Sensor  combinations 
were  considered,  but  most  of  the  effort  was  concentrated  on 
sensor  alarm  prioritizing,  then  assessment  cuing  and  image  proc¬ 
essing  were  added.  Common  methods  of  combining  sensors  alarms 
include  simple  "and,"  or  “or”  combinations,  and  a  more  compli¬ 
cated  "time  regulated  alarm  combination"  (TRAC) .  TRAC  requires 
multiple  alarms  on  the  same  sensor  or  on  multiple  sensors  in 
certain  time-period  combinations  before  an  alarm  is  reported. 

Assessment  Cuing 

Assessment  cuing  can  aid  the  operator  in  alarm  assessment  by 
providing  information  about  the  type  of  alarm  and  its  location. 
The  sensor  platform  can  then  be  aimed  at  the  alarm  location 
for  assessment  either  automatically  or  semiautomat ically .  In 
automatic  mode,  the  platform  responds  to  an  alarm  by  moving  until 
the  camera's  field-of-view  is  centered  about  the  alarm  azimuth. 

In  the  semiautomatic  mode,  the  controller  indicates  the  alarm 
azimuth  and  asks  the  operator  if  the  platform  is  to  be  turned  to 
that  bearing.  Upon  the  operator's  command,  the  platform  is 
turned  to  the  alarm  bearing  under  computer  control.  The  operator 
can  also  control  the  platform  in  a  totally  manual  mode. 

Currently,  the  acoustic  array  is  the  only  omnidirectional  intru¬ 
sion  sensor,  and  the  only  input  that  can  automatically  drive  the 
platform  to  an  azimuth  bearing  different  from  where  it  already 
points.  Detection  by  the  other  sensors  on  the  platform  is 
limited  to  their  f ields-of-view,  which  are  coincident  with  the 
camera's  field-of-view.  Additional  detectors,  such  as  perimeter 
sensors  from  a  facility's  Intrusion  Detection  System  (IDS),  could 
be  tied  into  the  RSS  as  other  sources  of  alarm  information  to 
drive  automatic  assessment. 

Image  Processing 

The  image-processing  system  was  explained  earlier  in  Section  2. 


Alarm  Prioritizing 

Alarm  prioritizing  received  most  of  our  attention  for  this  POP 
phase.  The  prioritizing  factors  considered  area: 

o  Weather 
o  Threat  analysis 
o  Location  of  alarm 
o  Alert  status 
o  Operations  schedule,  and 
o  Recent  alarm  history. 

Section  4  discusses  these  factors  in  detail. 

4.  ALARM  PRIORITIZING 

The  Weighting  and  Thresholding  Concept 

Alarm  prioritizing  methods  of  sensor  fusion  attempt  to  establish 
the  importance  of  each  sensor  alarm  through  a  concept  called 
"weighting  and  thresholding."  Only  alarms  with  enough  importance 
(weight)  are  presented  to  the  operator  to  reduce  the  number  of 
complex  judgments  the  operator  must  make  under  stress.  Alarm 
priorities  are  determined  by  considering  many  factors  whose 
interrelationships  may  be  slte-specif ic.  These  factors  must  have 
thresholds  established  that  can  be  based  on  measured  parameters 
like  temperature,  or  programmable  selectors  like  time  of  day  or 
location.  The  factor  thresholds  and  sensor  alarm  believability 
weights  (and,  therefore,  alarm  priorities)  will  change  with  time 
and  varying  conditions. 

The  Phase  1  RSS  demonstrates  the  weighting/thresholding  concept 
by  considering  weather  and  location  of  alarm  factors.  Each  alarm 
is  weighted  according  to  how  the  current  environmental  conditions 
affect  the  performance  of  the  alarming  sensor.  The  area  around 
the  pod  is  divided  into  10  wedges  of  36  degrees  each,  which 
radiate  outward  from  the  pod.  Each  wedge  is  assigned  a  threshold 
value  from  0  to  100,  that  reflects  the  relative  importance  of  the 
area  or  location  within  the  wedge.  The  threshold  values  are 
inversely  proportional  to  the  area's  importance.  The  sum  of  the 
weights  of  the  sensor  alarms  occurring  in  each  wedge  during  a 
given  time  period  is  compared  with  the  threshold  for  that  wedge. 
The  system  operator  is  notified  only  when  the  threshold  is 
exceeded.  For  example,  lower-weight  alarms  are  reported  in  a 
high-priority  wedge  (low  threshold)  rather  than  in  a  less 
important  (higher  threshold)  area.  Appendix  C  discusses  this 
weighting/threshold  approach  to  sensor  fusion  in  greater  detail. 


Weather 


The  effects  of  weather,  specifically  wind,  temperature,  rain, 
snow,  humidity,  and  light  level  on  individual  sensors  are  fairly 
well  known.  The  validity  of  sensor  alarms  can  be  judged  by 
noting  the  current  environmental  conditions  to  determine  if  any 
condition  exists  that  might  reduce  the  reliability  or  sensitivity 
of  a  particular  sensor.  For  instance,  high  winds  will  reduce  the 
accuracy  of  the  acoustic  array  target  bearing  determination;  and 
ambient  temperatures  of  85  to  95  degrees  F  significantly  reduce 
the  sensitivity  of  the  passive  IR  motion  sensor.  The  intelligent 
controller  automatically  adjusts  sensor  alarm  priorities  for 
prevailing  weather  conditions. 

Threat  Analysis 

Threat  analysis  is  another  factor  that  can  be  used  to  prioritize 
alarms.  Intelligence  sources  or  threat  analyses  may  provide 
information  such  as  location,  mode  of  transportation,  or  timing 
of  likely  intrusions.  With  this  knowledge,  the  appropriate 
sensors  can  be  brought  to  bear,  and  the  priority  of  these  sensors 
raised  to  hasten  detection  and  alert  the  operator.  By  knowing 
what  stimulates  each  type  of  intrusion  sensor  to  alarm,  one  can 
learn  something  about  an  intrusion.  For  instance,  if  the 
acoustic  array  identifies  an  acoustical  target,  the  target  is 
probably  mechanized  transportation  and  not  a  covert  intrusion. 

If  the  target  bearing  is  toward  a  critical  location,  it  should  be 
assessed  immediately  because  the  target  might  represent  an  overt 
intrusion.  But  if  only  motion  is  detected,  either  by  the  PIMS 
or  the  VMD,  it  is  likely  to  be  a  covert  intrusion.  The  PIMS' 
stimulus  consists  of  heat  sources  moving  across  its  narrow  field- 
of-view  (2  degrees)  out  to  about  150  meters,  while  the  VMD 
responds  to  meaningful  motion  in  any  direction  by  any  object. 

The  VMD  detection  zone  is  determined  by  the  camera  lens  and  the 
VMD  setup  and  almost  certainly  is  greater  than  the  PIMS  detection 
zone.  Thus,  even  though  the  camera  and  PIMS  are  aligned,  some 
possible  discrimination  of  range,  location,  and  alarm  stimulus 
may  aid  in  determining  the  type  and  location  of  threat. 

Location  of  Alarm 

The  location  of  alarms  is  another  way  to  prioritize  alarms.  The 
proximity  of  the  alarm  location  to  high-value  assets,  or 
the  ranking  of  critical  locations  according  to  asset  values  can 
establish  priorities.  When  alarm  locations  are  very  close  to 
important  assets,  and  if  the  threat  is  real,  it  means  the 
intruder  has  almost  reached  the  target,  and  the  response  forces 
should  act  immediately.  If  an  alarm  occurs  far  away  from 
potential  targets,  it  may  be  reasonable  to  wait  for  additional 


alarms  before  responding,  especially  if  there  are  significant 
delay  barriers  on  the  way  to  the  target.  On  the  other  hand,  if 
the  expected  target  is  very  vulnerable,  with  no  delay  barriers 
and  a  ’’soft”  enclosure,  all  alarms  must  be  considered  real  and 
response  time  minimized. 

Normally,  if  multiple  sensors/detectors  in  the  same  vicinity 
alarm,  then  the  validity  of  the  alarms  increases.  But  if 
security  sentries  or  patrols  are  in  the  area,  the  sensor  alarms 
can  probably  be  ignored  (and  may  even  be  caused  by  them) . 

Characteristics  of  the  site  geography  can  also  determine  alarm 
priorities.  Geographic  features  such  as  steep  hillsides  or  dense 
forest  act  as  delay  barriers  themselves,  so  alarms  outside  them 
are  not  so  important.  These  same  hills  (or  buildings)  may  pre¬ 
vent  timely  assessment,  however,  and  in  that  case,  the  response 
team  or  ground  patrols  should  be  alerted  immediately  for  alarm 
assessment  and  containment  of  the  threat.  The  RSS  intelligent 
controller  does  prioritize  alarms  by  location  by  assigning 
appropriate  thresholds  to  the  individual  wedges. 

Alert  Status 

The  base  or  facility  alert  status  effects  how  alarms  should  be 
handled.  Some  military  bases  use  3  levels  of  alert: 

o  Green  alert.  Things  are  normal  and  routine  security 
measures  are  in  order. 

o  Yellow  alert.  Security  awareness  is  increased  and  guard 
posts  and  ground  patrols  are  usually  reinforced. 

o  Red  alert.  This  is  the  highest  level  and  calls  for 
maximum  deployment  of  security  forces. 

As  more  sentries  and  patrols  are  deployed,  they  assume  detection 
and  assessment  duties,  and  sensors  in  the  same  areas  can  usually 
be  deactivated.  The  presence  of  additional  people  may  actually 
interfere  with  the  operation  of  the  IDS.  When  more  security 
forces  are  deployed,  they  will  probably  be  assigned  to  areas 
close  to  the  high-value  assets.  Intrusion  detection  emphasis 
using  sensors  would  then  be  shifted  farther  away  from  the  assets 
to  provide  earlier  warnings. 

The  intelligent  controller  could  be  programmed  to  automatically 
change  the  sensor  alarm  priorities  as  alert  levels  change.  This 
relieves  the  operator  of  the  responsibility  of  deciding  which 
sensors  are  important  at  any  particular  time. 
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Operations  schedule 

Alarms  can  be  prioritized  according  to  operations  schedules.  The 
RSS  capability  to  desensitize  specific  areas  permits  it  to  con¬ 
tinue  detection  and  assessment  at  important  locations  while 
ignoring  routine  sources  of  movement  or  noise.  Aircraft  runways 
or  motor  pools  are  common  sources  of  noise  that  could  be  masked 
to  the  omnidirectional  acoustic  array  while  it  continues  detec¬ 
tion  in  other  directions.  The  RSS  intelligent  controller  could 
sensitize  and  desensitize  both  locations  and  sensors  automatic¬ 
ally  according  to  the  operations  schedule,  thus  relieving  the 
operator  of  this  task. 

Permanent  IDS  sensors  occasionally  must  be  deactivated  for 
reasons  stated  above  or  to  allow  infrequent  activities  at  the 
facility.  For  example,  railroad  shipments  may  not  happen  often 
enough  to  justify  installing  permanent  security  sensors  to  main¬ 
tain  perimeter  security  in  the  presence  of  the  trains.  An  RSS 
could  be  used  to  maintain  vigilance  in  the  perimeter  sector  or 
zone  containing  the  railroad  tracks,  while  that  portion  of  the 
IDS  is  deactivated. 

Recent  Alarm  History 

Finally,  recent  alarm  history  can  be  used  to  prioritize  alarms. 
Successive  false  alarms  on  one  sensor  probably  indicates  a  mal¬ 
function.  The  controller  could  automatically  desensitize  the 
alarms  from  that  sensor  until  the  problem  is  corrected. 

If  several  intrusion  alarms  are  activated  in  a  sequence  that 
indicates  movement  toward  a  suspected  target,  then  the  intelli¬ 
gent  controller  could  report  this  real-time  intrusion  sequence  of 
alarms  as  a  high-priority  alarm  requiring  immediate  attention. 

5.  CONCLUSIONS 

The  FY87  effort  for  the  DNA  Robotics  for  Physical  Security  pro¬ 
ject  was  directed  towards  developing  a  proof  of  principle  Remote 
Security  Station  using  readily  available  hardware.  This  Phase  1 
effort  concentrated  on  developing  the  sensor  fusion  techniques 
used  to  reduce  the  false  alarm  rate  and  increase  the  probability 
of  detection.  Techniques  to  cue  the  system  operator  with  the 
location  of  alarms,  as  an  aid  in  assessment,  were  also  developed. 

The  Phase  1  effort  concentrated  on  developing  software  rather 
than  building  fieldable  hardware.  As  a  result,  the  Phase  l 
system  consisted  of  readily  available  hardware  and  little  atten¬ 
tion  was  paid  to  the  packaging  of  the  components.  The  basic 
algorithms  and  techniques  for  the  intelligent  combination  of 
sensor  information  were  developed,  although  field  testing  of  the 
system  needs  to  be  pursued  so  that  better  estimates  of  weighting 
factors  and  thresholds  can  be  obtained. 


The  RSS  consists  of  several  different  intrusion  detection  sensors 
utilizing  different  technologies.  By  using  a  variety  of  sensors 
that  are  adversely  effected  by  different  environmental  condi¬ 
tions,  there  should  always  be  at  least  one  sensor  providing 
reliable  alarm  data.  The  fusion  techniques  used  in  the  Phase  1 
system  exploit  this  concept  by  weighting  the  believability  of 
each  individual  sensor  in  a  summing  function  that  produces  "true" 
alarms.  Presently,  the  environmental  conditions  at  the  sensor 
platform's  location  are  used  to  alter  the  believability  of  alarms 
from  each  sensor.  This  should  reduce  the  number  of  nuisance 
alarms  caused  by  environmentally  induced  phenomenon.  Future 

?lans  include  adding  the  jpast  history  of  a  sensor's  performance 
nto  this  weighting  function. 

The  Remote  Security  Station  will  be  a  valuable  tool  for  security 
forces  at  both  fixed  and  temporary  sites.  It  has  the  character¬ 
istics  of  easy  deployment,  omnidirectional  coverage,  and  a 
variety  of  intrusion  detection  sensors  that  collectively  maintain 
reliable  alarm  information  under  all  environmental  conditions. 

6.  FUTURE  DEVELOPMENTS 

The  DNA  RSS  is  scheduled  for  Phase  2  development  during  FY88. 

This  effort  will  improve  upon  the  Phase  1  proof  of  principle 
hardware  to  yield  a  field  prototype  system.  While  the  Phase  2 
hardware  will  continue  to  be  based  on  a  stationary  tripod,  an 
eventual  upgrade  to  a  mobile  platform  is  being  considered  in  the 
design. 

Several  improvements  will  be  made  to  the  control  console  during 
Phase  2  development;  these  include  better  packaging,  an  improved 
operator's  interface,  and  improvements  in  the  fusion  of  sensor 
data. 

Rack  mount  video  and  computer  monitors  have  been  ordered  to 
replace  the  monitors  currently  in  use;  they  should  provide  a 
cleaner  installation  and  improve  the  appearance  of  the  operator's 
console.  The  new  computer  monitor  will  be  equipped  with  a  touch 
screen;  replacing  the  keyboard  as  the  operator's  interface  to  the 
system.  The  graphics  tablet  used  to  input  site  maps  for  the 
Phase  1  system  was  larger  than  necessary,  so  a  smaller  tablet 
that  will  mount  nicely  in  the  table  top  has  been  acquired.  New 
equipment  racks,  suitable  for  mounting  the  new  equipment  have 
also  been  ordered. 

As  mentioned  previously,  a  touch  screen  will  replace  the  keyboard 
as  the  operator's  interface.  Experience  with  other  security 
systems  has  shown  that  a  touch  screen  is  a  very  user  friendly 
device.  Rather  than  remembering  keystrokes,  the  operator  will 


now  simply  be  required  to  touch  the  proper  area  of  the  computer 
monitor.  The  menu  driven  graphics  make  it  easy  for  an  operator 
to  learn  the  system.  A  joystick  will  be  added  as  the  interface 
to  control  pod  motion  in  the  manual  mode.  In  addition  to  the 
manual  and  semiautomatic  modes  of  operation,  fully  automatic 
control  of  the  pod  will  be  integrated  this  year.  In  this  mode 
the  operator  will  be  required  for  assessment  only,  the  host  com¬ 
puter  will  detect  an  alarm,  point  the  pod  in  the  proper  direc¬ 
tion,  and  ask  the  operator  to  render  an  assessment. 

The  sensor  fusion  techniques  employed  in  the  Phase  1  system  were 
limited  to  establishing  wedge  thresholds  for  alarm  locations 
prioritizing  and  using  environmental  conditions  to  affect  a 
sensor's  believability  or  weight.  A  weighted  sum  of  all  sensors 
reporting  alarms  was  then  used  to  determine  real  alarms.  This 
method  is  intended  to  reduce  the  false  alarms  caused  by  effects 
of  the  weather.  No  field  testing  of  the  intrusion  sensors  was 
done  to  determine  exactly  how  they  were  affected  by  weather  con¬ 
ditions  or  the  optimal  weighting  factors  to  be  used  in  the  fusion 
algorithm.  The  environmental  dependencies  were  subjectively 
based  upon  the  past  experiences  of  sensor  experts  at  Sandia. 
Extensive  field  testing  of  the  RSS  intrusion  sensors  is  planned 
to  determine  the  proper  weighting  factors  for  each  of  the 
sensors.  Inclusion  of  the  past  history  of  each  sensor's  perform¬ 
ance  into  the  fusion  algorithm  is  also  planned  in  the  coming 
year.  This  should  reduce  the  number  of  false  and  nuisance  alarms 
produced  by  sensors  for  reasons  other  than  environmental 
conditions. 

The  Phase  1  pod  hardware  was  assembled  with  readily  available 
components.  No  attempt  was  made  to  make  the  system  ruggedized  or 
weather  proof.  The  Phase  2  effort  will  include  repackaging  the 
proof  of  principle  components  into  a  field  prototype  unit  that  is 
semi-rugged  and  weatherproof. 

Several  of  the  proof  of  principle  components  will  be  replaced  for 
the  field  prototype  pod.  The  Phase  1  pod  required  AC  power  to 
operate  the  pan/tilt  motors?  a  new  pan/tilt  platform  with  DC 
motors  has  been  acquired  so  that  battery  powered  operation  will 
be  possible.  A  new  directional  microphone  will  replace  the 
Bionic  Ear  on  the  upgraded  pod.  This  new  microphone  is  weather 
proof  and  should  exhibit  better  performance.  The  camera,  zoom 
lens,  and  IR  spotlight  are  all  susceptible  to  damage  by  moisture, 
so  environmental  enclosures  will  be  obtained  to  protect  these 
items . 

Improvements  to  the  acoustic  array  will  be  implemented  as  part  of 
the  Phase  2  development.  Many  nuisance  alarms  were  caused  by  air 
traffic  triggering  the  Phase  1  acoustic  array.  The  planned 
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enhancements  to  the  acoustic  array  will  enable  simple  classifica¬ 
tion  of  noise  sources,  so  that  differentiation  between  helicop¬ 
ters,  jets,  and  "other  sources"  can  be  attained.  This  should 
reduce  the  number  of  nuisance  alarms  originating  from  noise 
sources  such  as  established  air  traffic  routes. 

Improvements  will  be  made  to  the  setup  procedure  for  the  video 
motion  detector  (VMD) .  The  current  procedure  calls  for  setting 
several  parameters  that  require  an  experienced  operator  to  deter¬ 
mine  optimal  values.  This  setup  will  be  incorporated  in  the 
system  so  that  at  most  the  operator  will  be  requested  to  answer 
a  few  simple  questions.  The  computer  will  then  set  these 
parameters  for  the  current  conditions  based  upon  a  database  of 
knowledge.  The  operator  will  also  be  required  to  define  an  area 
of  interest  at  several  key  pan/tilt  locations.  The  remainder  of 
the  setup  will  be  taken  care  of  automatically  by  the  computer  and 
the  setup  parameters  will  be  dynamically  updated  with  changing 
weather  conditions  to  provide  optimal  operation  at  all  times. 

Two  new  sensors  will  be  evaluated  this  year  for  addition  to  the 
RSS  sensor  suite;  the  AN-PPS-15  Ground  Surveillance  Radar  (GSR) , 
and  the  Southwest  Microwave  model  #375A  monostatic  microwave 
sensor.  No  work  has  been  done  with  either  sensor,  but  plans  are 
to  evaluate  their  usefulness  and  incorporate  them  into  the  RSS 
sensor  suite  if  they  are  found  to  be  worthwhile  additions. 

The  Phase  1  pod  controller  and  weather  station  was  based  upon  the 
Sensor  Environment  Monitor  (SEM) .  Since  the  SEM  was  developed 
for  use  in  interior  environments,  it  was  never  intended  to 
measure  the  extreme  conditions  encountered  outdoors  and  the 
electronics  are  not  packaged  in  a  weather  proof  enclosure.  The 
Phase  II  effort  will  include  the  fabrication  of  a  new  weather 
station  and  pod  controller  that  is  designed  to  measure  exterior 
weather  conditions.  The  new  pod  controller  will  be  environment¬ 
ally  enclosed  so  that  it  is  suitable  for  deployment  in  exterior 
environments. 

Communication  to  the  RSS  sensor  platform  is  currently  achieved 
over  individual  copper  conductors  for  each  data,  video  and  audio 
signal.  Due  to  the  lossy  nature  of  the  transmission  of  these 
signals  over  copper,  the  distance  that  the  pod  can  be  deployed 
from  the  control  console  is  very  limited.  No  tests  have  been 
conducted  to  determine  the  limit  of  this  transmission  distance, 
but  it  is  on  the  order  of  several  hundred  meters  for  the  current 
system.  For  this  reason,  a  fiber  optic  communication  system  will 
be  used  for  the  Phase  2  RSS.  Individual  fibers  will  still  be 
used  for  transmitting  each  of  the  signals,  but  these  will  be 
jacketed  into  one  cable  so  that  deployment  will  be  easier. 

Three  copper  conductors  will  be  included  in  the  communication 
cable  for  the  purpose  of  power  transmission;  120  VAC  sent  over 
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these  conductors  will  be  used  to  power  a  no  VDC  supply  at  the 
sensor  pod.  This  DC  supply  will  be  used  for  nominal  power 
requirements  and  to  charge  a  battery  that  will  be  used  during 
peak  power  demands,  such  as  when  the  IR  spotlight  is  turned  on. 
The  fiber  optic  modules  are  capable  of  transmitting  signals  over 
a  distance  of  at  least  3  Km,  but  due  to  losses  in  power  trans¬ 
mission  the  remote  power  supply  will  only  operate  at  distances  of 
1  Km  or  less  from  its  120  VAC  source.  This  was  deemed  to  be  an 
acceptable  limit  to  the  distance  that  the  RSS  sensor  pod  could  be 
deployed  from  the  control  console. 

The  many  enhancements  to  the  Remote  Security  Station  planned  for 
FY#88  should  greatly  increase  the  system's  value  as  an  additional 
tool  to  be  used  by  security  forces.  Repackaging  of  the  com¬ 
ponents  will  produce  a  system  rugged  enough  to  be  deployed  in  the 
field.  Improvements  in  the  operator's  interface  will  make  the 
system  easier  to  use.  Finally,  improvements  in  the  sensor  fusion 
techniques  and  the  addition  of  more  capable  sensors  will  improve 
the  performance  of  the  system. 
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WEATHER  EFFECTS  ON  SENSOR  RANGE/SENSITIVITY 


RANGE  CHANGES  INVERSELY  WITH  PARAMETER 

REDUCE  PRIORITY  OF  ALARMS  AS  PARAMETER  INCREASES  UNTIL 

SOME  THRESHOLD  REACHED— THEN  DISREGARD  ALARMS. 


PRIORITIZING  ALARMS  BY  OPERATIONS  SCHEDULE 


Directionality  from  Passive  Sensors 
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DECLARE  AN  ALARM  AND  REPORT  THE  BEARING. 


CURRENT  PARAMETERS 


HARDWARE  AND  ALGORITHM  BLOCK  DIAGRAM 


CHANNEL  3 


RSS  SOFTWARE  OVERVIEW 


WEDGE  ALARM  THRESHOLDS  RANGE  FROM 
0  TO  100  INCLUSIVE.  SENSOR  WEIGHTS 
ALSO  RANGE  FROM  0  TO  100,  INCLUSIVE. 
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(SOUND) 


RSS  SOFTWARE  OVERVIEW 


CURRENT  CONDITIONS  29  OCT.  1987  CONTROLS  STATUS 

TEMPERATURE  20  DEG  C  1725: 58  OPERATION  MODE 


ZOOM  SPOTLIGHT 
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SEND  PAN  AND  TILT  COMMANDS.  IF  ANY. 
places  where  there  is  program-operator  interaction 
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SOFTWARE  OVERVIEW 


WEDGE  i - 1  WEDGE  1 


SENSOR  BEARING  ALGORITHMS 

There  are  many  similar  algorithms  which  can  be  used  to  determine  the  bearing  of 
an  acoustic  target.  One  common  method  is  based  on  the  difference  between  the 
Times-of-Arrival  (TOA)  of  a  far  field  signal  as  it  is  received  by  several 
sensors  positioned  at  known  locations  in  space.  The  calculations  needed  to 
determine  the  TOA  of  the  signals  involves  finding  the  cross-correlation  of  the 
different  sensor  signals.  Once  the  cross-correlation  of  the  signals  has  been 
determined,  the  delays  between  the  TOA  of  the  different  sensor  signals  can  be 
found  by  examining  the  cross -correlation  function  for  peaks.  It  is  the  delay 
between  the  sensor  signals  that  is  used  to  determine  the  bearing  of  the  acoustic 
target.  There  are  some  drawbacks  to  using  this  type  of  algorithm.  The  first  is 
that  a  large  number  of  calculations  are  needed  to  determine  all  the  cross¬ 
correlation  functions.  A  second  drawback,  which  is  inherent  in  the  calculation 
of  discrete  cross-correlation  functions,  is  that  only  discrete  values  for  the 
delays  can  be  obtained  ( i . e . ,  if  the  actual  delay  between  two  signals  was  4.4 
samples,  the  discrete  delay  would  be  4.0  samples).  This  results  in  an  inaccu¬ 
rate  estimate  of  the  bearing.  There  are  ways  to  increase  the  accuracy  of  the 
cross-correlation  algorithm,  but  they  involve  adding  more  sensors.  This,  then, 
increases  the  number  of  computations  needed  to  calculate  the  cross-correlation 
functions.  Also,  this  method  does  not  work  well  when  multiple  target  signals 
are  present  in  the  sensor  signals.  There  are  methods  using  the  cross-correla¬ 
tion  matrix  for  large  arrays  of  sensors  that  can  handle  multiple  targets. 
However,  these  methods  require  very  powerful  computers  to  handle  the  computa¬ 
tions  in  any  reasonable  amount  of  time. 


BEAMFORMLNfi  ALGORITHM 

Beamforming  is  in  many  ways  similar  to  cross-correlation,  but  differs  in  one 
important  way.  Instead  of  using  the  cross-correlation  function  to  calculate  the 
delays,  beam- forming  assumes  that  the  acoustic  source  of  interest  is  located  at 
a  specific  point  in  space  (the  horizontal  angle  to  the  assumed  point  in  space  is 
calTed  the  look-angle).  The  beamforming  algorithm  then  calculates  the  delays 
that  would  occur  if  the  source  was  actually  located  at  the  assumed  point  in 
space.  Using  these  delays,  the  algorithm  shifts  the  sensor  signals  by  an  amount 
opposite,  but  equal  to,  the  calculated  delays.  These  shifted  signals  are  then 
added  together  with  the  combined  signal  being  called  the  beamformer  output. 
Therefore,  the  beamformer  output  will  amplify  the  signal  of  a  target  when  it  is 
located  in  the  direction  of  the  look-angle,  and  cancels  the  signal  of  a  target 
when  it  is  not  in  the  direction  of  the  look-angle.  The  beamforming  algorithm 
uses  the  common  sine  function  to  preform  the  shifting  of  the  signals.  The  use 
of  the  sine  function  allows  the  algorithm  to  shift  the  signals  by  any  amount  of 
delay.  This  is  in  direct  contrast  to  the  cross-correlation  algorithm  which  can 
only  determine  the  delay  to  the  nearest  discrete  value. 


BEARING  ALGORITHM 

The  Initial  algorithm  used  with  the  DNA  Remote  Security  Station  is  a  spin-off  of 
the  beamforming  algorithm.  This  algorithm  involves  three  microphones  which  are 


B-l 


equally  spaced  around  the  circumference  of  a  circle.  The  microphones  are 
located  a  distance  d  from  each  other.  The  distance  d  between  the  microphones  is 
determined  by  the  following  equation, 


d  <  lambda  /  2  =  v  /  2*fh, 

where  lambda  is  the  wave-length  of  the  highest  frequency  of  interest,  v  is  the 
speed  of  sound,  and  fh  is  the  highest  frequency  of  interest.  This  distance  is 
used  so  that  180  degree  ambiguities  do  not  occur  in  the  beamformer  output. 

These  ambiguities  are  commonly  referred  to  as  spatial  aliasing.  After  sampling 
the  three  microphone  signals  (which  at  this  point  are  band-limited  so  that 
spatial  aliasing  does  not  occur),  the  algorithm  time  shifts  the  microphone  data 
for  twelve  different  look-angles.  These  angles  range  from  0-330  degrees,  with  a 
step-size  of  30  degrees.  The  30  degree  step-size  is  used  to  increase  the  speed 
of  the  algorithm,  and  produces  satisfactory  results.  The  time-shifted  signals 
are  then  added  together.  The  average  power  of  the  beamformer  output  is  then 
divided  by  the  average  power  of  the  three  microphone  signals.  This  ratio  is  an 
indication  of  the  amount  of  acoustic  power  in  the  look  direction,  compared  to 
the  total  acoustic  power  incident  on  the  three  microphones.  The  algorithm  then 
compares  this  ratio  data  to  a  predetermined  threshold.  If  the  ratio  data 
exceeds  this  threshold,  then  a  directional  noise  source  is  present.  The 
algorithm  then  determines  the  rough  bearing  of  the  noise  source  by  searching  for 
the  largest  value  in  the  ratio  data.  This  will  give  a  bearing  which  is  within 
approximately  +  15  degrees  of  the  target.  After  determining  the  rough  bearing, 
the  algorithm  then  computes  the  beamformer  output  and  ratio  for  look-angles  on 
either  side  of  the  rough  bearing.  Only  this  time,  the  data  used  for  these  cal¬ 
culations  has  a  wider  bandwidth.  This  wide-band  data  is  used  in  order  to 
improve  the  resolving  power  of  the  three  microphone  array.  It  is  important  to 
note  that  using  high  frequency  data  as  a  means  of  improving  the  resolving  power 
of  the  microphone  array  must  be  done  with  care  as  spatial  aliasing  can  occur. 
However,  because  the  algorithm  calculates  the  beamformer  output  and  ratio  over  a 
small  range  around  the  rough  bearing  (in  which  spatial  aliasing  cannot  occur), 
the  effects  of  spatial  aliasing  should  pose  no  problem.  Also,  the  look-angle  is 
varied  in  five  degree  increments  (which  should  give  the  needed  resolution  to 
point  a  camera  at  a  far  field  target).  A  block  diagram  of  the  algorithm  is 
shown  in  Figure  1. 


PROPOSED  HARDWARE 

The  hardware  used  to  run  the  target  detection  algorithm  is  based  on  a  VME  system 
with  a  68020  micro-processor  and  math  co-processor.  The  data  acquisition  system 
portion  of  the  hardware  consists  of  the  following:  1)  a  microphone  preamp  (see 
Figure  2);  2)  an  analog  line  amplifier;  3)  an  anti-aliasing  filter;  4)  a  sample- 
and-hold:  and  5)  a  12  bit  analog-to-digital  converter.  A  block  diagram  of  the 
hardware  is  shown  in  Figure  3. 


SYSTEM  LIMITATIONS 

There  are  three  major  limitations  which  will  effect  the  performance  of  the 
acoustic  bearing  algorithm.  These  limitations  are:  1)  range,  2)  tracking 
ability,  and  3)  bearing  accuracy. 
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FIGURE  1.  General  Block  Diagram  of  the  Proposed  Algorithm 
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FIGURE  3.  Block  Diaqraa  of  the  Proposed  Hardware 


Range  Limitations:  The  most  dominant  limitation  of  any  acoustic  system  is  its 
ability  to  "hear"  the  target  of  interest.  The  range  at  which  the  target  can  be 
heard  is  limited  by  three  major  factors.  These  factors  are:  1)  the  amount  of 
acoustic  energy  emitted,  2)  the  sensitivity  of  the  sensors  used  to  detect  the 
acoustic  energy,  and  3)  the  amount  of  background  noise  present  (both  acoustic 
and  system).  If  the  target  of  interest  is  a  jet,  which  radiates  large  amounts 
of  acoustic  energy,  then  detection  ranges  on  the  order  of  15-20Km  are  quite 
realistic.  If  the  target  is  a  helicopter,  reliable  detection  ranges  on  the 
order  of  5-10Km  would  probably  be  possible.  However,  for  typical  land  vehicles 
(which  do  not  emit  large  amounts  of  acoustic  energy),  detection  ranges  on  the 
order  of  100-700m  could  be  expected.  Longer  detection  ranges  would  be  possible 
if  the  vehicles  of  interest  were  tanks,  trains,  large  trucks,  and  other  large 
vehicles.  Unfortunately,  the  average  passenger  vehicle  is  very  quiet.  The 
amount  of  acoustic  energy  emitted  by  these  types  of  vehicles  is  very  low, 
especially  when  moving  at  slow  speeds.  Therefore,  the  detection  range  of  pas¬ 
senger  vehicles  will  probably  be  between  100m  and  300m  on  a  paved  road.  The 
detection  range  would  probably  increase  if  the  vehicle  was  traveling  on  a  dirt 
road  or  off  the  road.  The  reason  for  this  increase  is  due  to  the  increase  in 
road  noise  associated  with  traveling  on  dirt  roads,  and  the  increased  engine 
noise  associated  with  off-the-road  travel. 

Tracking  Limitation:  The  limitation  in  range  leads  to  another  important  limita¬ 
tion.  After  the  current  algorithm  detects  the  presence  of  an  acoustic  source, 
it  then  tracks  this  source,  giving  new  bearing  information  every  1.25  seconds 
(assuming  the  sound  level  of  the  source  persists).  The  algorithm  scans  bearings 
ranging  from  +/-15  degrees  from  the  last  known  bearing.  If  the  target  moves 
more  than  15  degrees  in  the  amount  of  time  it  takes  to  update  the  bearing 
estimate,  the  target  cannot  be  tracked.  To  place  this  in  perspective,  a  few 
examples  are  in  order.  Assume  that  a  sound  source  is  traveling  in  a  circle 
around  the  center  of  the  microphone  array.  If  the  radius  from  the  center  of  the 
microphones  to  the  source  is  100m,  then  the  maximum  speed  that  the  source  could 
have  around  the  circle  is  0.2094  rad/sec  (15deg/l .25sec)  which  at  100m  corres¬ 
ponds  to  a  speed  of  20.94m/sec  (or  46.85  mph).  This  is  the  theoretical  maximum 
speed  of  a  sound  source  at  100m  that  the  algorithm  could  track.  Of  course,  as 
the  radius  increases,  so  does  the  maximum  speed  at  which  the  sound  source  can 
travel  assuming  a  circular  path.  At  a  radius  of  500m,  for  example,  the  sound 
source  could  move  at  a  speed  of  105m/sec  (or  234  mph)  and  still  be  tracked. 
Obviously,  this  restriction  does  not  apply  if  the  sound  source  is  moving  towards 
the  microphone  array.  In  this  case,  the  angular  velocity  is  small. 

Bearing  Estimate  Accuracy  Limitations:  The  estimated  bearing  accuracy  is 
probably  the  least  significant  of  the  overall  system  limitations.  The  bearing 
estimates  for  strong  acoustic  sources  are  generally  not  more  then  +/•!>  degrees 
off.  In  fact,  the  error  is  usually  zero  degrees,  keeping  in  mind  that  by  design 
the  algorithms  bearing  estimates  are  in  increments  of  five  degrees.  Also,  the 
acoustic  source  used  to  obtain  these  estimates  was  static.  Of  course,  the 
accuracy  of  the  bearing  estimate  is  affected  by  background  noise  as  well  as 
atmospheric  conditions.  Wind  is  probably  the  most  detrimental  of  all  the  atmos¬ 
pheric  conditions.  Wind  is  a  source  of  background  noise  and  also  causes  errors 
to  occur  in  the  bearing  estimate.  For  example,  a  lOm/s  (22.3mph)  wind  which  is 
perpendicular  to  the  target  bearing  will  result  in  a  bearing  error  of  approxi¬ 
mately  2  degrees  (see  [2]).  Another  factor  that  will  effect  the  bearing 
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estimate  is  the  time-of-flight  of  the  sound  waves  emitted  by  an  acoustic  source. 
Sound  waves  travel  at  approximately  345m/seconds.  Therefore,  sound  emitted  from 
a  source  1km  from  the  microphone  array  takes  2.9  seconds  to  reach  the  micro¬ 
phones.  For  example,  if  an  acoustic  source  is  traveling  at  lOOm/sec  (224  mph) 
in  a  circle  around  the  microphone  array,  and  the  distance  to  the  source  is  1km, 
the  source  will  move  290m  in  the  time  it  takes  the  sound  waves  to  reach  the 
microphones.  This  translates  into  a  change  in  bearing  of  about  2.6  degrees. 

This  however,  will  not  be  a  problem  if  the  target  is  moving  toward  the  micro¬ 
phone  array,  or  if  the  speed  of  the  target  is  not  too  great.  All  of  these 
sources  of  error  combined  should  not  cause  a  significant  problem  if  queuing  a  TV 
camera  is  all  that  is  required  of  the  system. 
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AB8TRACT 

A  weighting/threshold-based  sensor  fusion  algorithm  to  decrease 
the  false  alarm  rate  (FAR)  while  maintaining  a  high  probability 
of  detection  (PD)  is  being  tested  in  the  Remote  Security  Station 
(RSS) .  The  RSS  is  being  developed  to  provide  temporary  intrusion 
detection  capability  on  short  notice.  It  consists  of  a  portable, 
multisensor  pod  connected  by  cable  to  a  manned  control  console. 
The  pod  is  set  up  outdoors  in  the  location  that  security  is 
needed;  the  console  and  operator  are  located  in  a  command  bunker 
up  to  a  kilometer  away.  The  RSS  software  filters  out  alarms  from 
low-believability  sensors  and  also  filters  out  alarms  in  low- 
priority  areas.  Each  sensor's  believability  is  proportionally 
encoded  as  a  weighting,  which  is  continually  updated  as  a 
function  of  the  environmental  conditions  affecting  that  sensor. 
Area  priority  is  proportionally  encoded  as  a  threshold  value  for 
each  pie-wedge  area  around  the  pod.  When  an  event  in  an  area 
triggers  one  or  more  sensors,  their  weightings  are  summed  and 
then  compared  to  the  area  threshold  value.  The  operator  is 
informed  of  the  event  only  if  the  summed  weighting  exceeds  the 
threshold.  Extensive  field  testing  has  not  yet  been  done,  but 
some  results  show  the  current  sensor  fusion  algorithm  decreases 
the  FAR  at  the  expense  of  lowering  the  PD.  To  increase  the  PD 
while  retaining  a  low  FAR,  the  weighting/  threshold  algorithm 
will  be  modified  to  use  temporal  data  and  pattern  recognition. 
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l . 0  introduction 

The  Remote  Security  Station  (RSS)  is  designed  to  provide 
temporary  intrusion  detection  capability  on  short  notice,  it  is 
most  suitable  for  replacing  or  augmenting  a  defective  portion  of 
the  perimeter  security  system  of  a  fixed  site,  although  it  may 
also  be  used  at  other  sites  such  as  new  airfields  or  stopped 
vehicle  convoys.  The  RSS  reduces  manpower  requirements  since 
only  a  single  operator  at  the  console  is  needed  to  run  the  RSS. 
The  RSS  sensor  pod  may  be  deployed  in  areas  or  under  conditions 
where  it  would  be  undesirable  to  station  a  man,  and  it  reduces 
operator  workload  by  filtering  out  some  false  alarms.  Sensor 
fusion  is  used  in  the  RSS  to  perform  this  filtering,  to  determine 
the  "truthfulness"  of  intrusion  detection  sensors  when  they 
trigger. 

2 . Q  Terminology 

In  the  following  discussions,  two  kinds  of  sensors  will  be 
referred  to.  The  first  is  the  intrusion  detection  sensor  which 
triggers  when  it  senses  an  event.  This  triggering  is  what  is  of 
the  most  interest,  although  some  sensors  also  output  an  analog 
measure  of  the  alarm  level.  The  second  type  of  sensor  is  an 
environmental  sensor,  which  continuously  outputs  a  measurement  of 
an  environmental  condition.  An  intrusion  detection  sensor  will 
be  referred  to  as  a  sensor  or  an  alarm  sensor,  whereas  an 
environmental  sensor  will  be  referred  to  explicitly  as  an 
environmental  sensor . 

Pwo  kinds  of  alarms  will  also  be  referred  to.  A  raw  alarm  is 
raised  when  an  alarm  sensor  has  been  triggered  by  an  event.  Raw 
alarms  are  received  by  the  control  program  only;  the  operator  is 
never  notified  of  raw  alarms.  A  true  alarm  is  raised  by  the 
control  program  after  it  has  evaluated  raw  alarms  and  determined 
that  an  intrusion  has  probably  occurred.  All  true  alarms  are 
relayed  to  the  operator  for  assessment. 

The  phrase  sensor  fusion  will  be  used  to  refer  to  two  related  but 
separate  tasks.  The  first  task  is  the  fusion  of  environmental 
information  with  the  alarm  sensor  information.  This  task  is 
performed  continuously  to  adjust  the  believability  measure 
(weighting)  of  the  sensors  and  to  keep  the  operator  apprised  of 
the  current  confidence  level  of  the  RSS.  The  second  sensor 
fusion  task  is  the  fusion  of  multiple  sensor  alarms  into  a  single 
value  which  determines  whether  or  not  the  operator  should  be 
alerted.  This  task  is  performed  only  when  there  have  been  one  or 
more  raw  alarms  within  the  past  iteration  of  the  control  program. 


3 . 0  Hardware 


The  RSS  consists  of  a  portable,  multi-sensor  pod  connected  by 
cable  to  a  manned  control  console  which  contains  an  IBM  PC  and 
other  processing  hardware.  An  environmental  monitoring  station 
is  also  set  up  near  the  pod  to  send  weather  information  back  to 
the  console.  The  pod  and  environmental  station  are  set  up  out¬ 
doors  in  the  location  that  security  is  needed,  while  the  console 
is  located  in  a  command  bunker  up  to  a  kilometer  away.  See 
Figure  1. 

R8S  SETUP 


SENSOR  CONTROL  CONSOLE 


ENVIRONMENTAL 

STATION 
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FIGURE  1 
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The  pod  and  the  console  are  major  hardware  units  without  which 
the  RSS  could  not  function.  The  RSS  could  still  operate  (at 
reduced  efficiency)  without  the  environmental  station.  The 
station  provides  information  with  which  sensor  confidence  is 
updated.  Should  the  environmental  station  be  unavailable,  there 
would  be  no  filtering  of  alarm  information  from  the  intrusion 
detection  sensors,  and  the  operator  would  be  alerted  to  all  raw 
alarms. 

A  more  detailed  description  of  each  of  these  three  units  follows 
(see  Table  1) . 


Pod 


Table  1.  Equipment  Lists 

|  Control  console  |  Environmental  station 


PIMS 

VMD 

Acoustic  array 
CCD  camera 
IR  spotlight 
Directional  mike 


PC-XT  computer 
Computer  monitor 
B/W  video  monitor 
Graphics  monitor 
Graphics  tablet 
Processors 


Thermometer 
Barometer 
Hygrometer 
Anemometer 
Photometer 
(Precip.  detector) 


3 . 1  Pod  and  Intrusion  Detection  Sensors 

The  system's  intrusion  detection  sensors  are  mounted  on  a  motor¬ 
ized  pan/tilt  assembly  on  a  tripod.  This  entire  unit  is  called 
the  pod.  Three  sensors  are  used  in  the  current  implementation: 
a  passive  infrared  motion  sensor  (PIMS) ,  a  video  motion  detector 
(VMD) ,  and  an  omnidirectional  acoustic  array. 

The  PIMS  is  an  infrared  device  which  senses  target  lateral 
motion.  It  compares  the  readings  received  in  its  two  side-by- 
side  detection  fields,  and  it  triggers  if  the  readings  differ. 

An  analog  output  from  the  PIMS  is  also  available  but  not  used  by 
the  RSS.  This  analog  level  is  a  measure  of  the  PIMS's  current 
activation.  The  PIMS  is  a  unidirectional  device  that  can 
register  events  only  in  the  direction  that  the  pan/tilt  assembly 
is  pointed.  The  field-of-view  (FOV)  of  the  PIMS  is  about  6 
degrees. 

The  Video  Motion  Detector  (VMD) ,  as  its  name  implies,  is  a 
visually-oriented  device.  It  processes  the  video  from  a  CCD 
camera  on  the  pod  and  determines  if  there  is  purposeful  motion 
(change)  across  preset  detection  zones.  The  VMD  is  also  a 
unidirectional  device,  as  it  receives  its  input  from  the  camera. 
The  camera's  FOV  depends  on  the  current  zoom  of  the  lens  and 
ranges  from  about  10  to  30  degrees. 
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The  acoustic  array  processes  the  signals  received  from  three 
microphones  placed  at  the  points  of  an  equilateral  triangle  at 
the  pod  location.  It  can  track  one  target  at  a  time  by  using  a 
beam-forming  algorithm  on  the  microphone  signals.  The  beam¬ 
forming  algorithm  compares  signal  strength  to  a  predetermined 
threshold  and  raises  an  alarm  when  the  threshold  is  exceeded. 

The  array  is  an  omnidirectional  sensor. 

Other  equipment  mounted  on  the  pod  includes  an  infrared  spotlight 
and  a  directional  microphone,  both  of  which  serve  as  assessment 
aids. 


3 . 2  Control  Console 

The  control  console  houses  an  IBM  PC-XT  computer,  a  color 
computer  monitor,  a  black-and-white  video  monitor,  a  graphics 
monitor,  a  graphics  tablet,  and  the  processors  for  the  acoustic 
array  and  the  VMD.  The  pod  control  program  runs  on  the  PC-XT. 

The  PC  monitor  is  used  to  display  status  information  while  the 
video  monitor  provides  video  to  the  operator  for  assessment 
purposes.  The  graphics  terminal  and  tablet  are  used  to  display 
site  maps  and  to  input  or  edit  these  maps. 

3 . 3  Environmental  Monitoring  Station 

This  station  is  a  package  of  environmental  sensors  which  include 
instruments  to  measure  temperature,  barometric  pressure, 
humidity,  wind  velocity,  and  light  level.  These  environmental 
sensors  were  part  of  an  existing  package  and  will  not  necessarily 
comprise  the  final  mix  when  the  RSS  is  fielded.  For  instance,  a 
precipitation  dectector  will  be  added,  while  the  barometer  and 
hygrometer  will  probably  be  deleted. 

The  information  from  the  environmental  sensors  is  relayed  to  the 
control  program  running  on  the  console  computer,  where  it  is  used 
by  the  sensor  fusion  code  to  adjust  the  weighting  of  the  intru¬ 
sion  detection  sensors. 

4 . o  Control  Program 

The  control  software  running  in  the  console  computer  serves 
several  functions.  First,  it  serves  as  the  interface  between  the 
human  operator  and  the  equipment.  The  software  relays  operator 
commands,  such  as  pan  or  tilt  directives,  to  the  pod  and  relays 
alarm  and  status  information  back  to  the  operator  at  the  console. 
Second,  the  software  serves  as  an  operator's  associate  which 
makes  a  pass  over  alarm  information  to  filter  out  false  alarms. 
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Within  the  control  program  is  a  main  loop  which  cycles  approxi¬ 
mately  four  times  each  second.  In  each  iteration  of  the  main 
loop,  the  control  program  performs  housekeeping  activities,  polls 
the  sensors,  updates  environmental  information,  and  updates  the 
sensor  weightings. 

5.0  Rationale  for  Weighting/Threshold  Scheme 

Whether  or  not  the  operator  is  ultimately  notified  of  an  alarm  is 
a  function  of  how  many  and  which  sensors  alarmed  and  in  which 
direction  they  registered.  In  a  security  system  which  does  not 
use  any  filtering,  ALL  alarms  would  be  relayed  to  the  operator. 
The  RSS,  however,  filters  some  alarms  if  they  are  raised  in  a  low 
priority  location  or  are  from  a  low-believability  sensor. 

Measures  of  area  priority  and  sensor  believability  are  maintained 
by  the  control  program  as  wedge  thresholds  and  sensor  weightings, 
respectively.  This  is  expanded  upon  in  the  following  sections. 

6 . o  Wedge  Thresholds 

The  area  around  the  pod  is  divided  into  a  number  of  wedges  which 
radiate  outward  from  the  pod's  location  (see  Figure  2) .  In  the 
current  control  program  implementation,  there  are  ten  uniform 
wedges,  each  spanning  36  degrees.  The  operator  assigns  to  each 
wedge  a  threshold  value  from  0  to  100,  inclusive,  to  reflect  the 
importance  of  that  area.  The  threshold  values  are  inversely 
proportional  to  an  area's  importance.  An  operator  can  therefore 
assign  values  of  nearly  100  to  mask  out  a  low- importance  area. 
Conversely,  he  may  assign  low  values  to  denote  a  high- importance 
area.  The  threshold  values  generally  remain  constant  while  the 
RSS  is  operating,  but  operator  may  adjust  them  if  necessary  in 
response  to  changing  conditions.  When  the  RSS  is  actually 
fielded,  the  allowable  range  of  the  threshold  values  may  be 
decreased  so  that  an  operator  can  never  completely  mask  out 
wedges  and  thereby  defeat  the  purpose  of  the  system.  Alterna¬ 
tively,  supervisors  may  be  the  only  ones  allowed  to  change  the 
thresholds. 

7.0  Sensor  Weightings  and  Weighting  Functions 

The  believability  of  a  given  alarm  sensor  is  represented  in  the 
control  program  as  a  weighting  for  that  sensor.  This  weighting 
is  proportional  to  the  sensor's  believability.  Environmental 
conditions  are  one  factor  which  influence  a  sensor's  believa¬ 
bility  as  an  alarm  received  from  a  sensor  under  adverse  weather 
conditions  is  more  likely  to  be  a  false  alarm  than  one  received 
under  ideal  conditions.  In  other  words,  as  environmental  con¬ 
ditions  degrade,  the  believability  of  sensors  also  decreases. 
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To  reflect  the  influence  of  weather  on  the  sensors,  the  control 
program  calculates  a  weighting  for  each  sensor  based  on  the 
current  environmental  conditions.  This  weighting  ranges  from  0 
to  100  inclusive  and  is  proportional  to  the  sensor's  believa- 
bility.  The  operator  cannot  adjust  a  sensor's  weighting. 

Associated  with  each  sensor  is  a  weighting  function  which 
calculates  the  sensor's  weighting  based  on  the  current  environ¬ 
mental  conditions.  Individual  functions  are  needed  for  each 
sensor  since  each  is  affected  by  different  environmental  con¬ 
ditions.  The  functions  currently  being  used  are  preliminary,  as 
very  little  field  testing  has  been  done  to  determine  the  exact 
effect  of  the  environmental  conditions  on  each  sensor.  As  a 
first  approximation,  common  sense  and  knowledge  of  the  way  each 
sensor  operates  were  used  to  write  simple  weighting  functions.  It 
must  be  stressed  that  the  current  functions  are  preliminary  and 
will  probably  be  changed  after  testing  is  completed.  Descrip¬ 
tions  of  the  weighting  functions  are  given  in  Table  2  and  in  the 
following  paragraphs. 


Table  2.  Sensor  Weighting  Functions 

Device  Function  parameters  Relation  of  weight  to  parameters 


PIMS  Temperature 

VMD  Light,  wind 

Array  Wind,  rain 

7.1  PIMS  Weighting  Function 


Inversely  proportional 
Proportional  to  light, 
inversely  proportional  to  wind 
Inversely  proportional 


Because  the  PIMS  is  an  infrared  device,  it  is  affected  adversely 
by  high  ambient  temperature.  The  weighting  function  for  the  PIMS 
is  of  the  form 


wtpijujs  =  f  (temperature) 

where  the  weighting  is  inversely  proportional  to  the  temperature. 
As  the  temperature  rises,  it  becomes  more  difficult  for  the  PIMS 
to  detect  the  difference  between  a  target's  temperature  and  the 
background  temperature.  Therefore,  as  the  ambient  temperature 
rises,  the  believability  of  the  PIMS  decreases  since  the 
probability  is  greater  that  an  alarm  has  a  spurious  cause. 

7.2  VMD  Weighting  Function 

The  VMD  weighting  function  is  of  the  form 

wtVMD  =  f (light,  wind) 


C-8 


where  the  weighting  is  proportional  to  light  level  and  inversely 
proportional  to  wind  velocity.  Low  light  level  affects  the 
performance  of  the  VMD  since  changes  in  the  scene  become  more 
difficult  to  detect  as  ambient  light  decreases.  High  light 
levels  are  not  a  p-oblem  since  the  automatic  iris  on  the  camera 
zoom  lens  preserves  high  contrast. 

7 . 3  Acoustic  Array  Weighting  Function 

The  RSS  does  not  have  a  noise  meter,  but  the  presence  of  some 
continuous  noise  may  be  inferred  by  examining  wind  velocity.  Wind 
blowing  across  the  microphones  will  confuse  the  array's  tracking 
algorithm.  The  form  of  the  weighting  function  for  the  array  is 
thus 


wtarray  =  f(wind,  rain) 

where  the  weighting  is  inversely  proportional  to  wind  velocity. 
Note  that  the  sound  of  falling  rain  can  similarly  confuse  the 
array,  and  so  a  precipitation  detector  will  be  included  in  the 
next  environmental  station. 

8.0  Raw  Alarms 

When  either  the  PIMS  or  the  VMD  is  triggered,  it  sends  a  single 
character  back  to  the  console  computer.  The  acoustic  array  sends 
back  a  number  indicating  the  compass  direction  in  which  it 
detected  a  target,  if  any.  (Alarm  direction  from  the  PIMS  and 
VMD  does  not  need  to  be  sent  because  these  two  sensors  are 
unidirectional  and  alarm  direction  is  the  same  as  current  pan 
angle) .  A  sensor  sends  information  to  the  console  computer  only 
if  the  sensor  has  been  triggered.  At  each  iteration,  the  control 
program  polls  all  three  sensors  for  raw  alarms.  If  there  have 
been  any,  the  control  program  initiates  the  determination  of 
whether  the  operator  should  be  apprised  of  the  situation. 

9.0  Action  Upon  Receiving  Raw  Alarms 

When  one  or  more  raw  alarms  are  received,  the  control  program 
determines  if  the  operator  should  be  notified  of  the  event. 

Sensor  fusion  is  used  to  merge  the  information  of  multiple 
alarms.  For  each  wedge,  the  following  is  calculated: 

wttotal  =  Wj_  +  W2  +  W3  +  ...  +  Wn 

where  the  W^  are  the  weightings  of  the  sensors  that  triggered  in 
this  wedge.  This  sum  is  then  compared  to  the  threshold  value  for 
that  wedge.  If  the  sum  exceeds  the  threshold,  the  control  pro¬ 
gram  raises  a  true  alarm  and  the  operator  is  notified.  If  the 


sum  is  below  threshold,  then  no  alarm  is  raised  and  the  operator 
is  not  notified  of  the  event.  The  sum  of  weightir  s  is  used  for 
ease  in  combining  the  information  from  more  than  c  e  sensor,  on 
the  principle  that  two  or  more  heads  are  better  than  one;  i.e., 
if  more  than  one  sensor  triggers  in  a  given  wedge,  the  proba¬ 
bility  is  greater  that  this  is  a  true  alarm. 

When  the  operator  is  notified  of  a  true  alarm,  he  must  view  the 
location  of  the  event  and  render  an  assessment.  The  three 
possible  alarm  assessments  are  Real,  Nuisance,  and  Unknown.  Real 
alarms  are  those  which  have  discernible,  threatening  causes,  such 
as  a  column  of  armored  vehicles.  A  Nuisance  alarm  also  has  a 
discernible  cause,  but  it  is  nonthreatening  like  a  rabbit. 

Alarms  classified  as  Unknown  have  no  apparent  cause  and  may 
indicate  a  hardware  failure.  Alarm  statistics  are  stored  each 
time  the  operator  makes  an  assessment.  This  information  includes 
the  assessment  rendered,  sensor (s)  and  wedge  involved,  wedge 
number,  strength  of  alarm  (total  weighting) ,  and  the  time  of  the 
event.  Temporal  information  such  as  this  will  be  used  in  later 
versions  of  the  weighting  functions. 

10.0  Results  and  Discussion 

Since  the  RSS  is  still  in  the  developmental  stage,  little  testing 
has  been  done.  It  appears  that  the  FAR  is  decreased  by  masking 
out  wedges  (giving  them  high  threshold  values) .  However,  the  PD 
is  consequently  decreased  since  all  alarms  are  masked  out  before 
they  can  be  processed. 

The  RSS  must  undergo  extensive  testing  under  diverse  weather 
conditions  before  any  conclusions  may  be  drawn  about  the  accuracy 
of  the  weighting  functions  in  calculating  sensor  believability . 

It  is  expected  that  the  functions  will  be  an  effective  means  of 
filtering  out  false  alarms.  Withrespect  to  the  fusion  of  the 
information  of  multiple  alarms,  it  seems  that  the  weighting/ 
threshold  algorithm  is  effective  in  combining  the  results  of  two 
or  more  sensors  which  trigger.  For  instance,  the  VMD  was 
observed  to  trigger  occasionally  on  some  random  distortion  in  the 
video  image.  If  the  wedge  threshold  is  set  to  a  high  value,  then 
the  raw  alarm  will  not  have  enough  weight  to  raise  a  true  alarm, 
and  this  false  alarm  is  filtered.  The  PIMS  triggers  on  the  same 
sort  of  phenomenon  that  the  VMD  does  (change  in  scene  due  to 
motion) ,  but  it  relies  on  infrared  information.  When  both  the 
VMD  and  the  PIMS  triggered  at  the  same  time  there  was  usually  an 
actual  event  as  "sightings"  were  made  in  both  the  visual  and 
thermal  spectra.  The  sum  of  the  weightings  of  the  VMD  and  PIMS 
usually  exceeded  the  wedge  threshold,  and  a  true  alarm  was  raised 
as  desired.  Triggering  on  an  event  from  different  sensors  is  an 
excellent  indication  of  a  true  alarm,  and  the  weightincf/threshold 
effectively  represents  this  by  the  summation  of  the  weightings  of 
all  the  raw  alarms  in  an  area. 


11.0  Future  Work/Recommendations 

With  respect  to  sensor  fusion  in  the  RSS,  future  work  may  be 
aimed  in  two  directions:  writing  new  weighting  functions  which 
take  advantage  of  field-test  results  and  temporal  information; 
and  use  of  code  and  specialized  hardware  for  recognizing  patterns 
in  the  sensor  data. 

11.1  New  Weighting  Functions 

Extensive  testing  of  the  RSS  must  be  done  under  diverse  weather 
conditions.  This  is  of  major  importance  for  refining  the  weight¬ 
ing  functions  to  accurately  calculate  the  believability  of 
sensors  in  various  environments. 

The  believability  of  a  sensor  is  based  not  only  on  the  current 
environmental  conditions,  but  also  on  the  performance  history  of 
the  sensor.  The  weighting  functions  will  be  modified  to  consult 
temporal  information  (stored  alarm  statistics)  to  find  patterns 
and  trends. 

11.2  Pattern  Recognition 

Some  improvement  in  performance  may  be  gained  by  using  new 
weighting  functions.  However,  such  modification  can  increase  the 
RSS  performance  only  up  to  a  certain  point.  As  the  functions 
become  more  complicated,  a  greater  the  percentage  of  processor 
time  will  be  spent  executing  them. 

This  amount  of  time  could  become  prohibitively  great  if  the 
number  of  sensors  deployed  on  the  pod  is  increased  as  expected. 
Also,  there  exist  history  patterns  which  cannot  be  discerned  in 
any  algorithmic  way.  Upgrading  the  hardware,  such  as  by  employing 
a  parallel  architecture,  can  solve  the  problem  of  processor  load. 
However,  the  pattern  recognition  problem  requires  a  completely 
different  approach  in  software.  In  general,  the  RSS  produces  a 
single  output  from  a  large  number  of  inputs,  a  number  which  will 
increase  when  temporal  information  is  used  and  more  sensors  are 
added.  This  type  of  problem  is  ideal  for  solution  by  a  neural 
network.  Therefore,  to  increase  RSS  performance  considerably,  a 
neural  network  implementation  may  be  investigated  to  augment  the 
weighting/threshold  sensor  fusion  algorithm. 
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